Semi-supervised natural language acquisition

Natural Language processing (NLP) is a field that combines linguistics, cognitive science, statistical machine learning and other computer science areas in order to compile intelligent computer systems that can understand human languages. NLP has various applications, among which are machine translation, question answering and search engines. The field of NLP has, in the past two decades, come to simultaneously rely on and challenge the field of machine learning. Statistical methods now dominate NLP, and have moved the field forward substantially, opening up new possibilities for the exploitation of data in developing NLP components and applications. Many state of the art natural language algorithms are based on supervised learning techniques. In this type of learning, a corpus consisting of texts annotated by human experts is compiled and used to train a learning algorithm. While supervised learning has made substantial contribution to NLP, it faces some significant challenges. Many fundamental NLP tasks, such as syntactic parsing, part-of-speech (POS) tagging and machine translation, involve structured prediction and sequential labeling. For such kind of tasks, compiling annotated corpora is costly and error prone due to the complex nature of annotation. I refer to this challenge as the annotation bottleneck. A closely related challenge is that of domain adaptation. Supervised algorithms usually perform well when the training data and the data for which they should provide predictions (the test data) are drawn from similar domains. When an algorithm trained with data from one domain is to provide predictions for data taken from a substantially different domain, its performance markedly degrades. Creating corpora for every test domain is not feasible due to the aforementioned annotation bottleneck. Supervised natural language learning is also challenged by methodological problems. Annotation schemes for tasks such as syntactic parsing are often based on arbitrary decisions. Such schemes often provide a detailed description of certain structures while addressing others only briefly. Many applications would benefit from different annotation decisions. This work focuses on developing machine learning techniques that deal with these challenges. Its main theme is utilizing the plentiful amounts of raw text available nowadays, for creating state of the art algorithms that use little to no manually annotated data. Cases where little amounts of manually annotated text is utilized are referred to as semi-supervised learning; while cases where only raw text is used are known as unsupervised learning. This work explores semi-supervised and unsupervised techniques for structured